Sampling from a larger population

Image from: https://danawanzer.github.io/stats-with-jamovi/

My background

• Cognitive neuroscience / experimental psychology.

• Social perception and cognition.

• I am not a statistician. (Did I say that already?)

My background

My undergrad and postgrad stats classes looked like this:

• Each week described a different statistical test.

• Your job was to choose the right test for a given type of data and run the test (usually via point-and-click in SPSS).

• Then you interpret the p-value.

• Easy, right?

My background

In the wake of the reproducibility crisis:

• I felt the need to become more statistically literate.

• Enter:

  • Richard McElreath’s excellent textbook (McElreath 2020)
  • Solomon Kurz’s brilliant translation into tidyverse principles (Kurz 2023).
  • Various papers and books by Andrew Gelman.

Champagne inference on a beer budget

A quote from Andrew Gelman (Gelman, 2024):

once the data have been collected, the most important decisions have already been done

Champagne inference on a beer budget

All statistical models are fundamentally limited and need to be framed within the wider scientific context (McElreath 2020), such as …:

Champagne inference on a beer budget

Before we make inferences and draw conclusions, we should spend more time (Scheel et al. 2021):

• Forming concepts.
• Developing valid measures.
• Identifying boundary conditions and auxillary assumptions.
• And so on…

Caution (!) is required

• Statistical inference is not magical.
• Inferences rest on a lot of assumptions and data quality.
• Inferences are likely to be fragile/tentative/suggestive.
• So, be cautious!
• And try to create research designs that do not rely too heavily on one particular part of your inferential model.

Towards statistical thinking

• Develop a statistical philosophy rather than rely on historical statistical rituals (Gigerenzer 2018).
• There are many different approaches to the same question.
• Have a sense of how your approach fits into the mix of options, in terms of pros and cons.
• Be able to defend and justify your choices explicitly.

Single-level linear regression

Common statistical tests are forms of linear regression

https://lindeloev.github.io/tests-as-linear/

Homework

• Go away, get some data and run single level regressions in R.
• Use different types of data - continuous and categorical predictors etc.
• I bet there are a million tutorials online.
• Aim: understand the basics forwards, backwards and sideways - as the rest builds on this core foundational knowledge.

Multi-level linear regression

Multi-level regression: why bother?

• One general and versatile way to approach data analysis.
• It avoids the obsession with picking the “right” statistical test.
• There is an active community of users and lots of resources available to guide you.
• It is suitable for a wide range of data in psychology and neuroscience.

Summary vs trial-level data

Partial pooling or shrinkage

let’s build in an intuition

Partial pooling or shrinkage

let’s plot it

Fixed and varying effects

## specify the model formula with only fixed or population-level terms
formula = dv ~ 1 + condition 

## and now add varying intercepts per participant
formula = dv ~ 1 + condition +
          (1 | participant)

## and now add varying intercepts and slopes per participant
formula = dv ~ 1 + condition +
          (1 + condition | participant)

Estimation approaches - basics

Adapted from Kruschke & Liddell, 2018

Estimation approaches - parameters

Estimation approaches - wider reading

Science needs more David Bowie

Thanks for your attention

And thanks to these fine folks:

• John Bartlett for his tutorial on reproducible presentations in R (this is solid gold).

• Lisa Debruine for sharing lots of example presentations

And here’s my stuff

• Slides: www.rich-ramsey.github.io/talks/sbs-retreat-25/

• Code: https://github.com/rich-ramsey/talks

• Website: www.rich-ramsey.com

• Github: https://github.com/rich-ramsey